Evaluating and measuring your programmes? Top 4 valuable lessons from Communiqué Award Judges
Jeremy Clark, Co-Chair of the Judges for the Communiqué Awards; Founder and Chairman of Clark Health Communications.
The Communiqué Awards have been around for 24 years, and strong evaluation and measurement has always been a key part of what it takes to win one of these prestigious awards.
But more than that, it’s an area where the awards have really worked to embed and reward best practice across our industry. The improvements and progress in how we evaluate the outcomes of the work that we do have been significant in that time BUT this is a continuing challenge and there is still much to do.
With the launch of the 2023 Communiqué Awards this week, and as the Healthcare Communications Association’s new Evaluation Standards and Best Practice Committee begins its work, we wanted to share some initial thoughts on this topic.
So, from the perspective of our industry’s leading awards programme, here are the top 4 things you should be thinking of when you’ve finished implementing your communications or education programme and you are just pulling the report together.
No! Wrong. That is the first key recommendation.
1. Think evaluation from the very start. You must be thinking of evaluation and measurement when you are conceiving, designing, developing, agreeing the budget, getting sign off, implementing and then reporting on your communications and education programmes. From an awards perspective the judges can always tell the campaigns that have genuinely set out to achieve certain outcomes (we’ll come back to this key word later!) and put metrics in place in advance to measure that (sometimes very simple ones). And we can spot those entries, thankfully fewer these days, that implement an activity and build up a case for impact only once it is complete.
Why does this matter? Well, for a start you are unlikely to win an award. Honestly. But more importantly as an industry we need to be doing work that is focused on making a genuine difference; a change in patient behaviour, an update to medical practice or a change in health policy. Knowing what this is and how we are going to measure it at the start is critical for this.
This isn’t just an issue of impact, or of agencies or in-house team delivering a return on investment, or of winning awards. From our point of view, it’s an issue of industry reputation.
2. Know where you are starting from. When planning any campaign or programme you must have some clear data showing where you are starting from. With no start point, no meaningful benchmark, it’s almost impossible to demonstrate any impact or real patient or professional outcome.
In the Communiqué Award entry criteria we say, “We work and operate in a world where there is a wealth of data and insights, so there is really no excuse for projects and programmes that are planned without this data”. Can you imagine one of the major FMCG companies investing in a campaign for one of their brands without a clear idea of what element of consumer perception or behaviour they are seeking to change, and what the quantified start point is? And yet, in award entries, we often see very little or even zero information used as a clear, quantified benchmark against which impact and success will be measured.
Every healthcare or pharmaceutical company has access to a wide range of data sources and have sophisticated and robust internal plans and business cases for their own work. So, work with your colleagues and/or clients to find and review this information and use it as the solid foundations for your campaign planning and, of course, the benchmark for your evaluation.
3. Be objective about objectives. We give quite a bit of guidance in our Communiqué entry kit about what the judges want to see in the ‘objectives’ section of the entries they review, and recently we have awarded marks specifically for objective setting. It should be simple right? But we still see some entries with vague and unconvincing objectives.
Entrants sometimes tie themselves up in knots with tortuous long paragraphs or they describe strategies or even tactics. It’s hard to avoid the conclusion that some teams simply didn’t know what they were trying to achieve when they started a project or that the objectives seemed to change mid-project.
We just want short, ideally SMART, bullet points explaining what impact they were aiming to have, ideally at the level of a patient or HCP behaviour change or health outcome with a goal that is built around the benchmark data described above. Again, I hope it’s obvious how important this is for all campaign and programme planning, not just for the purpose of entering awards.
4. Focus on measuring ‘outcomes’ not just activity. Even if there are some clearly written objectives it sometimes turns out that they focus mainly on ‘doing stuff’: “Our goal was to deliver a great meeting” or “engage with our audience”. These can be helpful outputs to measure that provide important insights into the audience response and help improve implementation, but the danger is that only these things then get evaluated with no attempt to show a real impact.
As described above, we would consider a strong outcome to be a measurable and sustained change in patient behaviour or outcome, an update to professional practice or a change in health policy. However, the key thing is to be clear what the need is in your specific situation and set a goal around that.
Once you have set a clear ‘outcome’ focused objective, the measurement of that outcome does require thought and planning from the very start of the project or programme, it can’t really be bolted on at the end. It may also require a bit of innovation in what measures are used, through what channels and at what point in the project. There seems to be an idea that measuring outcomes is automatically more difficult, and perhaps most significantly, more expensive than measuring other things. It is true that some budget should always be ringfenced to support evaluation, however, in our award entries we are starting to see some elegantly simple and inexpensive ways of measuring outcomes. It’s great when we see entries where the same level of thought and creativity has been applied to measurement approaches as to the big campaign idea and we will look to share some examples shortly.
The HCA Evaluation Standards and Best Practice Committee recognises that there are remaining challenges and opportunities in these four topics, and some further key issues beyond this top four. Education and training within our teams will be key and we will be announcing in the new year how we will aim to support learning and best practice sharing in this vitally important area.
In the meantime, 2023 Communiqué Awards are open for entry with key information available here and the closing date in March. Hopefully, these thoughts will help in preparing your entries.